[throw] the baby out with the bathwater

to lose the good alongside the bad in an attempt to dispose of the bad

The idiom "throw the baby out with the bathwater" warns against getting rid of something valuable or important while trying to eliminate something undesirable. It originated in the 16th century when communal bathing was more common, and family members would use the same bathwater in succession. The baby, being the last to bathe, was at risk of being accidentally discarded with the dirty water. This phrase is used to caution against rejecting the good along with the bad.

Are you sure that we are not throwing the baby out with the bathwater in dealing with this?
Janet quitting her job because her ex started working there is a classic example of throwing out the baby with the bathwater.
Just because you don’t get what you want from people does not mean you should end communication with them and therefore throw out the baby with the bathwater.
While making changes to the project, be careful not to throw the baby out with the bathwater by discarding valuable ideas in the process.
In revising the company's policies, they need to ensure they don't throw the baby out with the bathwater, losing important elements of the existing structure.